With the Med-PaLM series of LLMs Google is one of the few companies you can claim expertise in building medical domain specific LLMs. The latest addition has been AMIE (Articulate Medical Intelligence Explorer).
AMIE is a conversational medical AI optimized for diagnostic dialogue. AMIE is instruction fine-tuned with a combination of real-world and simulated medical dialogues, alongside a diverse set of medical reasoning, question answering, and summarization datasets.
AMIE has a self-play based simulated dialogue environment with automated feedback mechanisms to scale its capabilities across various medical contexts and specialities. There are two types of self-play loops in place : 1. An “inner” self-play loop, where AMIE leveraged in-context critic feedback to refine its behavior on simulated conversations with an AI patient agent; 2. An “outer” self-play loop where the set of refined simulated dialogues were incorporated into subsequent fine-tuning iterations.
During online inference, AMIE used a chain-of-reasoning (COR) strategy to progressively refine its response conditioned on the current conversation to arrive at an accurate and grounded reply to the patient in each dialogue turn
Across multiple axes corresponding to both specialist physician (28 out of 32) and patient actor (24 out of 26) perspective, AMIE was rated as superior to PCPs while being non-inferior on the rest. However, the results should be interpreted with appropriate caution. Translating from this limited scope of experimental, towards real-world tools, requires significant additional research and development.